Quantization matrix has been utilized in various video coding standards, such as MEPG2 and H.264/AVC to improve the subjective video quality. Each of transform coefficients is quantized according to an individual quantization step as specified by the quantization matrix (or scaling matrix). Due to its effectiveness, the quantization matrix has also been adopted by the emerging High Efficiency Video Coding (HEVC) system. The quantization matrix usually has the same dimension as the transform used by the system. For example, in MEPG2 and H.264/AVC, the transform sizes include 8×8 and 4×4. Therefore, the sizes of quantization matrix also support 4×4 and 8×8. Default quantization matrices as well as user-defined quantization matrices are used in H.264/AVC. Furthermore, separate quantization matrices are selected for Intra mode and Inter mode in H.264/AVC. The quantization matrices for Intra 4×4, Inter 4×4, Intra 8×8 and Inter 8×8 are shown in FIGS. 1A-D respectively. The encoder can decide whether the current picture will use a flat quantization matrix, default quantization matrix or user defined quantization matrix. If a user-defined quantization matrix is selected, information associated with the quantization matrix has to be incorporated in the bitstream so that the decoder can recover the corresponding quantization matrix.
In MEPG2 and H.264/AVC, the maximum size of quantization matrix is 8×8. The memory requirement for storing default quantization matrix is relatively small and it won't pose any significant issue to the coding system. When a user defined quantization matrix is used, the data size associated with the user defined quantization matrix is not a big issue either. Nevertheless, in HEVC, more transform sizes have been adopted and the transform size can be as large as 32×32. For example, to store two (Intra and Inter) 4×4 default quantization matrices and two 8×8 default quantization matrices only require 160 bytes. Using the same way to store two 32×32 default quantization matrices alone will require 2048 bytes. The total storage requirements of quantization matrices for MPEG2/H264/AVC and HEVC are shown in Table 1 and Table 2 respectively. As shown in the tables, the data size requirement for the quantization matrices in HEVC is increased substantially (more than 18 times). For high-speed hardware-based implementation, the quantization matrices may be hardwired and the data size associated with the quantization matrices will have significant impact on the device. Therefore, it is very desirable to develop a new quantization matrix representation scheme that can reduce the data size required to store the quantization matrices. When user-defined quantization matrices are used, large quantization matrices may consume sizeable precious bitrate. Therefore, it is also desirable to develop a new quantization matrix representation scheme that can reduce the data rate for transmitting the user-defined quantization matrices.
TABLE 1Memory SizeMatrix sizeNum of valuesPU TypeComponent(byte)4 × 416Intra/InterY/Cb/Cr968 × 864Intra/InterY128Total224
TABLE 2NumMemory SizeMatrix sizeof valuesPU TypeComponent(byte)4 × 416Intra/InterY/Cb/Cr968 × 864Intra/InterY/Cb/Cr38416 × 16256Intra/InterY/Cb/Cr153632 × 321024Intra/InterY2048Total4064(1814%)